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1.  OVERVIEW  OF  STUDIES 


In  thl>  paper  we  raise  the  lasues:  what  aak.es  for  effective*  diagnosis 
and  remediation  of  linear  algebraic  equations,  and  how  do  theae  lasues 
relate  to  the  development  of  Intelligent  tutoring  systems?  As  a  basis 
for  this  discussion  we  report  two  structured  Interviews  and  a  case 
study.  After  Introducing  aach  study,  we  discuss  each  under  the  head* 
Ings  of  diagnosis  and  remediation. 

Introduction  to  Study  A:  the  first  structured  Interview.  Three  male 
teachers  of  algebra  and  one  female  mathematics  teacher's  aide,  all  from 
school  districts  ir.  the  San  Franciscw  area,  served  sb  subjects.  They 
had  each  taught  algebra  for  between  10  and  40  years. 

We  presented  each.  Individually,  with  a  copy  of  a  questionnaire  (see 
Appendix  A)  of  algebra  tasks  and  a  student's  incorrect  solutions  to  the 
several  tasks,  and  asked  them  to  diagnose  the  student's  error (s)  for 
each  Item  and  to  suggest  remediation.  They  were  free  both  to  spend  as 
much  time  as  they  wished  on  any  Item  and  to  look  over  any  set  of  Items 
to  check  for  patterns  In  the  students'  solutions.  (Please  note  that 
the  teachers  did  not  have  students  present). 

We  compared  their  diagnoses  to  the  known  Incorrect  procedures  used  by 
the  students  (determined  from  Interview  work,  Sleeman,  1986),  and 
analysed  their  suggestions  for  remediation. 

Introduction  to  Study  The  second  structured  Interview:  In  order  to 

see  If  the  approaches  and  opinions  of  the  teachers  in  Studies  A  and  C  of 

this  paper,  and  of  Slseman,  Kelly  and  Grant  (1985),  generalised  to  a 

*We  studied  the  behaviour  of  experienced  teachers  diag¬ 
nosing  and  remediating  and  assumed  that  because  of 
their  experience,  they  were  effective.  No  attempt  was 
made  to  measure  their  effectiveness  objectively. 


different  culture  and  echool  system,  the  first  author  Interviewed  three 
Irish  secondary  school  mathematics  teachers.  Two  of  the  teachers  had 
over  20  years  of  experience  and  the  third  had  over  4;  each  teacher  had 
experience  teaching  algebra  to  all  7  years  In  the  school.  The  inter¬ 
view  was  based  on  the  questionnaire  used  In  Study  A  (see  Appendix  A)  and 
around  the  issues  that  arose  fron  that  study. 

Introduction  to  Study  £:  a.  case  study  In  remediation.  In  Study  C,  an 
algebra  teacher  fron  a  high  school  In  San  Francisco  area  was  observed 
while  reaediatlng  the  algebra  errors  of  eight  students.  The  algebra 
errors  in  this  study  were  diagnosed  by  the  PIXIE  prograa  (Sleeman,  1982) 
and  were  available  to  the  teacher  as  a  basis  for  his  reaedlstlon.  The 
teacher  was  shown  how  to  Interpret  the  printout.  The  goal  of  the  study 
was  to  abstract  a  model  of  reaedlstlon  from  this  teacher's  interaction 
with  the  several  (8)  students.  While  the  focus  of  study  C  was  remedia¬ 
tion,  we  learned  aoae  laportant  facets  about  diagnosis  from  it.  A  lim¬ 
itation  of  this  study  Is  that  a  post-test  was  not  possible  due  to  time¬ 
tabling  difficulties  at  the  school. 

We  shall  now  discuss  the  three  studies,  first  under  the  heading  of  diag¬ 


nosis  and  then  remediation. 


2.  DIAGNOSIS 


Diagnosing  the  "whet"  of  algebra  errors  -  A  means-ends  search. 

An  analysis  of  the  protocols  of  Study  A  suggests  two  major  findings  con¬ 
cerning  the  teachers'  diagnostic  strategy:  a)  that  the  teachers  used 

L 

,  a,  (GPS)  Ceneral  Problem  Solver-type,  means-ends  search  (Newell  A  Simon, 

l  1963),  and  b)  that  searching  for  patterns  across  items  Is  a  powerful 

heuristic  for  diagnosing  known  student  errors. 

A  CPS-llke  algorithm  reasons  forward  from  the  Initial  state  of  the  prob¬ 
lem  towards  the  goal  state  reducing  the  'difference*  between  them  by 
»  testing  appropriate  Intermediary  steps  (Newell  A  Simon,  1963).  The  evl- 

ik 

dence  for  a  GPS-type  search  In  the  present  atudy  was  strr  gly  suggested 
j  by  the  protocols  of  the  majority  of  the  teachers.  Figure  1  gives  an 

example  of  such  a  search  tree: 


Insert  Figure  1  about  here 


The  teacher  whose  protocol  la  summarised  In  figure  1  made  three  attempts 
to  reduce  the  'difference'  between  the  equation  and  the  student's  solu¬ 
tion  by  choosing  substeps  (nodes  M,N,0;  nodes  M,P  and  nodes  M,Q,R,S). 
For  example,  he  seemed  to  believe  that  43/7  (node  S)  was  not  'close 
enough'  to  11  to  make  that  path  plausible.  He  finally  opted  for  the 
route  M-T-U-V-Z.  (Note  that  the  teacher  appears  to  have  taken  a  large 
step  to  "move"  from  nodes  V  to  Z). 

In  Study  B,  the  Irish  teachers  showed  evidence  for  a  similar  mean6-ends 
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search  in  diagnosis.  A  typical  statement  noted  with  this  task  vas: 
"How  did  he  get  from  there  (the  equation)  to  there  (the  solution)? 
Let's  see,  he  eight  have  subtracted  the  5...  No  that  won't  work.  Maybe 
he  divided  by  the  3,  and  then  added  2?"  etc. 

Not  ell  teachers  were  practiced  at  this  aethod  of  diagnosis,  thus  soae 
items  were  left  partially  diagnosed  or  undiagnosed.  In  fact,  both 
groups  considered  the  task  of  diagnosis,  as  presented,  'artificial'*  and 
stressed  that  they  would  Insist  on  seeing  all  of  the  student's  workings 
of  a  task.  However,  even  under  those  conditions  students  will  some¬ 
times  turn  In  Incomplete  workings  of  tasks,  In  which  case  some  form  of 
means-ends  analysis  would  be  necessary  for  a  complete  diagnosis. 

In  these  studies  we  presented  teachers  with  a  series  of  equation- 
student-answer  pairs  and  asked  them  to  make  diagnoses.  It  might  rea¬ 
sonably  be  argued  that  these  teachers  had  no  options  but  to  perform  a 
means-ends  search.  However,  below  we  list  several  other  possibilities 
which  could  be  used  by  a  diagnostician,  namely: 

1.  Given  the  Initial  form  of  the  equation  (e.g.,  one  containing  brack¬ 
ets)  and  the  form  of  the  correct  answer  (say  an  Improper  fraction) 
teachers  might  create  a  set  of  anticipated  answers. 

2.  Recall  the  set  of  most  frequently  made  errors  In  a  domain  and  check 
if  any  of  these  would  explain  the  observed  student  errors. 

3.  Given  the  tssk,  can  the  student's  answer  be  achieved  by  manipulat¬ 
ing  the  coefficients  in  the  equation  by  all  known  operators  (a  variant 
on  the  means-ends  guidance  approach). 

4.  Create  the  correct  solution  path,  and  subsequently  include  all 
known  Incorrect  versions  of  the  rules  and  Incorrect  orderings  (this  Is 


what  PIXIE  does),  and  check  If  any  of  these  explain  the  student's 
answer. 

and  the  default: 

5.  If  no  diagnosis  could  be  found  (by  using  any  of  the  methods  includ¬ 
ing  means-ends  guidance)  then  conclude  that  the  student  needs  to  be 
rataught  (parts  of)  the  skill. 

Searching  for  patterns  of  errors.  In  Study  A,  it  was  generally  the  case 
that  the  teachers  (one  in  particular)  who  searched  for  patterns  across 
items  were  the  ones  who  were  more  successful  at  diagnosis,  but,  of 
course,  searching  for  patterns  did  not  guarantee  finding  then. 

In  our  analysis,  we  labelled  an  attempt  to  diagnosis  in  which  the 
teacher  found  at  least  one  complete  path  to  the  solution  (l.e.,  gave  a 
full  set  of  steps  to  explain  a  student's  error  as  a  complete  search;  we 
labelled  other  attempts  incomplete  searches).  For  example,  the  items  in 
Set  2  (See  Appendix  A)  were  of  the  general  form  ax  ■  bx  +  c  with  the 
student's  incorrect  solution  x  "  (a  +  b  +  c)/2.  Several  teachers  who 
gave  incomplete  searches  stopped  when  they  reached  the  simplification  of 
the  solution  in  the  general  form  x  ■  a  +  b  +  c,  unable  to  explain  the 
student's  division  by  2.  (In  this  case,  the  student  had  incorrectly 
added  all  the  coefficients,  and  divided  by  2  because  there  were  two  X- 
terms  in  the  original  equation  (Sleeman  1986)). 

We  further  sub-classlfled  "complete"  searches  as  matching  searches 
(those  that  matched  the  known  error  patterns  of  the  students  (from  Slee- 
aan,  1986))  and  alternative  searches  (those  that  were  plausible  alterna¬ 
tive  explanations  for  the  error(s)).  From  these  a  metric  for  a 
teacher's  skill  at  finding  the  students'  known  error  patterns  was 
developed  (see  Table  1). 
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Insert  Table  1  about  here 


Overall,  the  teachera  gave  complete  aeerchea  for  an  average  of  16  of  the 
23  Items,  gave  Incomplete  searches  for  an  average  of  3  Items,  and 
offered  no  diagnosis  whatsoever  for  an  average  of  3  Items.  (Items  with 
no  diagnosis  were  classed  by  the  teachers  as  "unusual " .  For  exemple, 
the  Items  In  Set  4  were  difficult  for  three  of  the  teachera  because  the 
student's  solutions  contain  two  different  values  for  X,  even  though  the 
equation  in  each  case  la  linear). 

For  example,  Tea  .her  4  had  the  highest  percentage  of  both  complete  and 
matching  searches. 

The  average  number  of  matching  searches  was  just  under  5  for  the  remain¬ 
ing  three  teachers.  For  Teacher  4,  it  was  thirteen  -  due  in  part  to  his 
skill  In  finding  common  error  patterns  in  a  set.  He  would,  typically, 
diagnose  the  error  of  the  first  Item  and  check  If  this  diagnosis  also 
explained  the  errors  in  the  remaining  ltem6  In  the  net.  If  not,  he 
would  iterate  this  process  until  he  found  a  common  error  pattern,  or 
determined  to  his  satisfaction  that  there  was  none.  He  searched  per¬ 
sistently  for  a  pattern,  moreover,  he  did  not  allow  himself  to  be 
daunted  by  the  absence  of  a  pattern  underlying  the  Items  in  Set  3. 
Consequently,  he  found  the  patterns  In  Sets  4  and  5  —  underscoring  the 
Importance  of  continuing  to  search  for  patterns  of  errors  in  the  face  of 
dlsconfirmlng  evidence. 

On  the  other  hand,  teachers  who  typically  looked  at  diagnosis  on  a  per 
Item  basis  usually  gave  Incomplete  searches,  or  terminated  the  search 
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when  £  feasible  solution  path  was  found  (l.e.,  an  'alternative  search'). 
Such  behaviour  may  lead  to  the  reporting  of  soae  superficial  solution 
paths.  For  example,  a  student's  solution  of  X  *  6  for  the  equation  3  + 
4X  -  18  was  explained  as  the  student's  'forgetting'  the  '+4'  and  solving 
the  remaining  equation,  3X  ■  18.  (In  this  case  the  student's  error  was 
actually  the  result  of  misinterpreting  the  plus  sign  as  a  multiply  sign 
(which  gave  12X  “  18),  and  then  subtracting  12  from  both  sides.) 

Diagnosing  the  'why'  of  algebra  errors  -  taking  diagnosis  a  step  deeper 

The  teacher  In  Study  C  (l.e.,  the  one  who  tutored  eight  students  based 
on  PIXIE's  diagnoses)  believed  that  many  different  causes  underlie  what 
appears,  superficially,  to  be  the  same  error.  He  searched  for  a  causal 
underpinning  (a  why)  f  .  the  syntactical  error  (the  what ) .  For  him, 
diagnosis  was  not  complete  without  a  causal  explanation.  This  can  be 
Illustrated  by  how  the  teacher  handled  three  students  who  displayed  the 
same  mal-rule  namely.  Inverting  the  final  fractional  solution.  This 
teacher  concluded  It  was  unfamlllarlty  with  Improper  fractions  for  two 
students;  for  the  third  It  was  s  misunderstanding  of  the  mathematical 
notation  for  expressing  fractions. 

His  method  of  diagnosis  was  to  present  the  student  with  a  simplified 
version  of  the  equation,  observe  the  method  used,  and  from  this  Informa¬ 
tion  Infer  s  reason  for  the  error  noted.  For  example,  he  Inferred 
errors  due  to:  a)  an  algebraic  procedure  with  limited  application;  b) 
misunderstanding  fractional  notatlon-A,  c)  misunderstanding  fractional 
notatlon-B;  and  d)  unfamlllarlty  with  Improper  fractions. 

a)  Errors  due  to  an  algebraic  procedure  with  limited  application: 

In  this  class  of  errors,  the  teacher  concluded  that  the  error  Identified 
by  PIXIE  was  caused  by  the  student  using  an  algebraic  procedure  that  had 
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only  Halted  app’  ‘.cation*  It  worke-*  for  alaple  equations,  but  caused 
errors  in  aor.  difficult  ones. 

For  example,  In  the  case  of  atudent  A3,  PIXIE  indicated  the  atudent  was 
having  difficulties  with  equations  of  the  fora  3X  ■  S.  To  ascertain 
the  reason  behind  the  error,  the  teacher  set  the  atudent  the  alapler 
equation  3X  ■  6  (alapler  because  It  gives  an  Integer  solution).  The 
atudent  solved  it  by  saying  "3  tlaes  2  la  6 . "  From  this,  the  teacher 
concluded  that  the  student  could  sot  solve  an  equation  like  3X  “  5, 
since  the  student  couldn't  think  of  what  nuaber  tlaes  3  gives  3.  The 
procedure  of  looking  for  a  whole  nuaber  to  substitute  for  X  In  order  to 
balance  both  sides  of  the  equation  had  only  Halted  application  and 
foundered  when  the  value  for  X  was  a  non-integer.  The  teacher  verbal¬ 
ised  his  reasoning  about  the  error: 

(For  the  equation  6X  ■  12,  he  said:)  "Soae  people  ask:  '6  tlaes  what 
nuaber  Is  12?'.  When  you  are  used  to  doing  It  this  way,  it  Is  hard  to 
do  It  when  you  are  dealing  with  a  fraction.  You  need  a  method  that 
will  always  work." 

b)  Errors  due  to  misunderstanding  fractional  notatlon-A: 

One  student  appeared  to  have  a  general  procedure  for  solving  an  equa¬ 
tion,  but  because  of  unfaal llarlty  with  fractions  had  difficulty 
expressing  the  solution.  For  exaaple,  given  the  equation  9X  “  6,  his 
solution  was  X  •  1.  He  could  perform  the  rule  "divide  each  side  by  3" 
and  obtain  3X  ■  2.  However,  confused  by  the  format  for  fractional  solu¬ 
tions  (2/3),  he  Instead  subtracted  the  2  from  the  3.  The  teacher 
pointed  out  that  the  student's  problem  was  net  p.imarlly  algebraic,  but 
mistaking  the  notation  of  fractions  (for  that  of  subtraction).  (And 
supposedly  following  the  subtraction  bug  of  "subtract  the  smaller  number 
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from  the  bigger"  (Brown  end  Burton,  197B)). 


c)  Errors  due  to  misunderstanding  f rsctlonsl  notation-B. 

In  this  error,  the  teacher  believed  the  student  knew  how  to  solve  an 
equation  of  the  fora  aX  •  b  by  dividing  both  aides  of  the  equation  by  £, 
but  wrote  the  final  solution  as  a/b,  due  to  a  misunderstanding  of  the 
mathematical  notation  of  the  fraction. 

For  example,  the  teacher  set  student  A2  the  equation  3X  •  -2.  He  wrote 
X  “  -3/2.  The  teacher  asked  the  student  how  he  would  represent  '5 
divided  by  4."  He  wrote  "4/5." 

d)  Errors  due  to  an  unfamlllarlty  wl ch  improper  fractions 

In  this  class  of  errors,  the  teacher  believed  th..  student  to  have  a  gen¬ 
eral  procedure  for  obtaining  the  solution,  but,  being  unfamiliar  or 
uncomfortable  with  improper  fractions,  the  student  expressed  the  solu¬ 
tion  as  a  proper  fraction  Instead.  For  example  (student  Al),  the  equa¬ 
tion  vas  9X  »  16  to  which  the  student  gave  the  solution  X  •  9/16.  The 
teacher  responded  "You  are  used  to  getting  fractions  less  than  1.  You 
might  want  to  write  X  -  9/16,  just  because  it  look6  better  (than  16/9)''. 
This  type  of  error  might  also  Indicate  the  student's  tendency  to  regress 
to  earlier  methods  when  faced  with  new  problems  (Davis,  Jockusch  6 
McKnight,  1978). 

Summary:  Diagnosis 

To  summarise  this  section  on  diagnosis ,  we  can  see  that  under  conditions 
of  limited  information,  teachers  are  likely  to  use  a  means-ends  search 
to  discover  the  incorrect  solution  path  of  the  student,  and  that  a  ten¬ 
dency  to  look  for  error  patterns,  if  consistently  applied,  often  leads 
to  good  diagnosis.  We  have  also  seen  that  diagnosis  of  error  paths  may 


be  Just  half  the  story  -  determining  the  reason  for  the  error  may  also 
be  Important. 


3.  DECIDING  ON  APPROPRIATE  REMEDIATION 

The  need  for  a  detailed  diagnosis  to  serve  as  a  basis  for  remediation  Is 
a  basic  assumption  underlying  much  of  the  work  In  Intelligent  tutoring 
systems  -  e.g.  DEBUGGY  (Brown  &  Burton,  1978)  and  LMS/PIXIE  (Sleeman, 
1982).  However,  many  teachers'  approaches  to  remediation  may  not  be 
guided  by  this  perspective. 

The  teachers  In  Study  A  sometimes  suggested  remediation  for  fewer  errors 
than  they  diagnosed  (see  Table  2). 


Insert  Table  2  about  here 


The  teachers  appeared  to  review  their  diagnoses  and  make  some  summary 
judgement  about  the  student's  difficulties  and  then  begin  remediation. 
They  rarely  remediated  more  than  one  Issue  per  taBk,  and  their  sugges¬ 
tions  ranged  from  vague  statements  that  advised  the  student  to  go  'Back 
to  basics,'  through  procedural  prescriptions  like  'Get  rid  of  added 
things  first,'  to  conceptual  calls  to  'Show  the  student  the  difference 
between  Xs  and  numbers.'  Generally  speaking,  the  teachers  favoured 
rule-based  over  'conceptual'  remediation  (see  Table  3). 
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Insert  Table  3  about  here 


Only  one  of  the  four  teachers  consistently  referenced  the  current  alge¬ 
bra  Item  In  his  remediation  -  the  others  gave  more  general  feedback  (aee 
Table  4). 


Insert  Table  4  about  here 


Some  diagnosed  errors  say  have  been  Ignored  because  these  did  not  fal 
under  the  scope  of  an  agenda  triggered  by  the  task  (Putnam,  1987). 
According  to  Putnam  (1987),  a  sample  of  second-grade  teachers  working 
with  both  actual  and  simulated  students  did  not  probe  for  a  detailed 
diagnosis  before  they  began  to  reteach  the  topic.  Thus,  reteachlng  was 
often  at  a  general  level,  and  did  not  always  reference  the  task  on  which 
the  student  had  encountered  difficulty.  Putnam  suggests  that  such 
teachers  are  following  some  script-based  agenda  and  that  a  perceived 
weakness  In  an  area  "triggers"  part  of  that  agenda.  Alternatively, 
these  errors  may  not  have  been  judged  as  critical  for  understanding 
algebra. 

The  teachers  In  Study  B  said  that  they  would  remediate  fewer  errors  than 
they  had  diagnosed  for  motivational  reasons,  psrtlcularly  In  cases  of 
low-ability  students.  They  believed  that  pointing  out  successes  In 
mathematics  was  more  effective  In  the  long  run  to  pointing  out  failures 
(for  related  reading,  see  Fennema  and  Be!.r,  1980;  Kulm,  1980;  Reyes, 
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1980) 


The  teachers  In  Study  B  further  believed  that  tlae  spent  on  diagnosing 
errors  due  to  violation  of  rules  Bight  be  better  spent  remediating  what 
they  considered  to  be  aore  fundamental  problems  In  algebra:  considering 
the  X  variable  to  be  a  'letter'  (Indicating  major  conceptual  misunder¬ 
standings  In  mathematics  —  see  Davis,  1984;  Kuchemann,  1978), 
misunderstanding  of  fractions  and  negative  numbers,  misapplying  earlier 
knowledge  (Davis,  Jockusch  4  McKnlght,  1978),  and  not  knowing  basic 
mathematical  facts. 

\ 

Whet  we  noted,  therefore,  among  these  teachers  wa6  a  tendency  not  to 
develop  a  detailed  diagnosis  before  beginning  remediation. 

Study  C:  The  case  study 

The  teacher  In  Study  C  seems  to  be  an  exception  to  the  above  trend  In 
that  this  teacher's  remediation  was  based  not  only  on  the  diagnosis  of 
the  syntax  of  the  error,  but  also  on  the  reason  behind  the  error.  His 
approach  to  remediation  can  be  outlined  as  follows: 

After  diagnosing  the  what  and  the  why  of  the  error,  the  teacher: 

a)  Referred  Indirectly  to  the  error. 

b)  Reaffirmed  the  correct  procedure.  (Note:  the  teacher  did  not 
explicitly  Indicate  when  to  apply  this  procedure). 

c)  Reassured  the  student  that  the  new  method  gave  acceptable  solutions. 

d)  Cave  additional  Instruction. 

e)  Gave  practice  Items. 

We  shall  now  discuss  each  of  these  steps  In  some  more  detail. 


12 


a)  Referring  Indirectly  to  the  error : 

The  teacher  pointed  out  the  students'  errors  In  an  Indirect  fashion  (as 
If  not  lessen  the  students'  Motivation  for  Mathematics  c.f.,  the  teach¬ 
ers  In  Study  B).  The  teacher  In  Study  C  used  techniques  such  as  a> 
uolversallslng  the  error  by  claiming  that  it  was  common,  (e.g.,  to  stu¬ 
dent  A2,  who  worked  3X  ■  2  as  X  ■  3/2,  he  said  "You  Make  a  common  ml  s* 
take  that  Many  people  do").  (This  tendency  to  uni vernalise  the  error  Is 
Interesting,  since  the  uni vernalisation  of  errors  Is  known  to  be  one  of 
the  curative  factors  of  group  psychotherapy  (Yalom,  1980)).  b)  Other 
techniques  Included  remaining  tentative  In  assigning  blame  to  the  stu¬ 
dent  for  the  error  (e.g.  for  the  student  who  wrote  X  ■  9/16  for  9X  -  16, 
the  teacher  aald,  "You  might  w^nt  to  write  X  »  9/16,  Just  because  it 
looks  better"),  or  c)  Incriminating  some  undefined  "others"  for  It 
(e.g.,  for  the  student  who  wrote  X  -  3/2  for  3X  -  2,  the  teacher  said, 
"Would  you  believe  that  some  people  would  write  3/2?"),  and  d)  allowing 
the  student  an  excuse  for  the  behaviour  (e.g.,  to  one  student  he  said, 
"You  had  some  problems  with  this  (6X  “  9).  It  seems  that  you  subtracted 
6  from  9.  You  might  have  been  confused.  It  doesn't  work.  It  might 
look  like  a  good  answer"). 

b)  Reaffirming  the  correct  procedure.  Once  the  teacher  had  pointed  out 
an  error  to  the  student,  he  set  about  reaffirming  the  correct  procedure. 
For  example,  to  student  A4  he  ssld,  “The  procedure  of  dividing  across  by 
3  in  3X  •  6  should  be  the  same  for  3X  ■  5,  no  matter  if  you  get  nice 
numbers  or  not." 


To  student  Al,  concerning  an  equation  of  the  form  aX  -  b,  he  said,  “Even 
if  this  number  £  Is  bigger  than  this  number  a,  the  procedure  (dividing 
both  sides  by  a)  Is  still  the  ssme." 


c)  Reassuring  the  student  that  the  new  Method  gives  eccepteble  aolu- 
tlons.  The  teacher  typically  reassured  the  students  that  the  'unusual' 
solutions  produced  by  the  now-reaf firmed  correct  procedures  were  indeed 
acceptable. 

To  the  student  (A2)  uncomfortable  with  fractional  solutions,  he  said, 
"Don't  let  It  (the  fraction)  bother  you.  Two-thirds  Is  s  good  nuaber. 
There  are  a  lot  of  fractions  In  the  world."  To  those  uncoafortable  with 
improper  fractions  (ag,  student  A4),  he  said,  "9/6  Is  a  nuaber  (the 
solution  to  6X  -  9).  Is  It  a  whole  nuaber?  No.  But  It  Is  still  a 
nuaber.  This  Is  a  legal  nuaber.  These  nuabers  exist  In  the  world. 
We  don't  give  up  because  the  answer  Is  a  fraction.  Let's  figure  out  the 
best  way  to  write  this  nuaber  (reduces  It  to  3/2).  This  Is  not  a  nice 
nuaber.  The  nuaerator  on  top  Is  bigger  than  the  denoalnator.  Frac¬ 
tions  coae  In  all  shapes." 

d)  Giving  additional  Instruction.  If  It  were  called  for,  the  teacher 
presented  new  material  during  reaedlatlon.  For  example ,  he  explicitly 
labelled  fractions  "proper"  or  "Improper".  He  showed  some  students  how 
a  fraction  e/b  (expressed  here  In  a  general  fora)  was  elaply  another  way 
of  writing  arb;  showed  another  student  that  division  was  really  a  case 
of  repeated  eubtractlon.  And,  to  a  final  student,  two  procedures  for 
solving  the  saae  aquation  (dividing  by  the  X  coefficient  or  multiplying 
by  Its  Inverse). 

a)  Giving  practice  lteas.  The  teacher  In  Study  C  gave  only  three  of  the 
eight  students  practice  lteas  (one  to  each  student).  The  Itaa  required 
the  student  to  demonstrate  a  grasp  of  the  reaffirmed  procedure.  The 
nuaber  of  practice  lteas  assigned  In  this  study  was  saall,  but  each  of 
the  three  teachers  In  Study  B  stressed  the  Importance  of  giving  many 
practice  lteas  to  each  student.  In  fact,  they  believed  that  one  error 
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was  enough  for  the  student  to  handle  at  a  tlae  and  would  assign  a  full 
aet  of  exaaples  to  drive  one  point  home  (a  slallar  approach  was  recoa- 
aended  by  Buckingham,  1933).  A  series  of  studies  need  to  be  undertaken 
which  atteapts  to  determine  the  laportance  of  these  several  steps  for 
Reaediatlon. 

Rule-based  vs.  "conceptual"  instruction  or  relnstructlon 

Perhaps  the  approach  to  diagnosis  and  reaediatlon  would  differ  markedly 
if  algebra  was  taught  conceptually  (rather  than  procedurally  as  in  these 
studies).  A  classic  division  of  instructional  approaches  aay  be  labelled 
the  "conceptual  vs  rule-based"  division,  which  has  fueled  debate  on 
Instruction  in  mathematics  since  at  least  the  turn  of  the  century  (e.g., 
Beatley,  1954;  Byers,  1980;  Cronbach  A  Snow,  3977;  Davis,  1984; 
Elsenberg,  1975;  Godfrey,  1910;  Ormell,  1976;  Skeap,  1976). 

The  eaphasls  on  using  rules  to  tutor  In  algebra  was  noted,  with  soae 
exceptions,  in  the  structured  interviews,  in  the  case  study,  and  in 
Sleeaan,  Kelly  and  Grant  (1985).  In  Study  B,  the  three  teachers  indi¬ 
cated  that  the  students  who  required  extensive  reaediatlon  in  algebra 
were  ones  for  whoa  a  conceptual  reaediatlon  was  inappropriate.  They 
felt  that  such  low-ability  students  were  better  served,  in  the  tlae 
allowed  within  an  exaa-orlented  eystea,  by  being  given  a  snail  nuaber  of 
hard-and-fast  rules. 

These  teachers  noted  additional  advantages  to  a  rule-based  approach 
including  relatively  straightforward  structuring  of  lesson  units  and 
individual  lesson  plans.  Using  a  procedural  approach,  one  had  a  finite 
set  of  rules  to  teach  together  with  a  aechanlcal  directive  to  "do  the 
aaae  thing  to  both  sides"  which  was  seen  as  "easier  to  teach"  than  the 
■any  ■etaphors,  illustrations,  etc.  required  in  a  conceptual  approach. 


(see  alto  Sleeoan  at  al.,  1985).  Finally,  the  teachers  In  Study  B 
believed  that  "auccess"  in  teaching  was  easily  measured  when  based  on 
the  learning  of  rules.  One  could  discover  what  rules  had  and  had  not 
been  learned.  Remediation  and  further  Instruction  within  this  fraae- 
work  were  then  ‘‘clear*. 

Difficulties  steaming  froa  the  wide  range  of  Individual  differences 
aaong  their  students  was  a  concern  for  both  the  American  and  Irish 
teachers  (on  the  topic  of  Individual  differences  and  mathematics,  see 
Carry,  1983;  Cronbech  4  Snow,  1977;  Fenneaa  &  Behr,  1980;  Snow,  1963; 
Threadglll-Sovder,  1985).  For  example,  an  Irish  teacher  cited  Plage- 
tlan  research  to  argue  that  many  students  were  Ill-prepared  for 
conceptual-based  Instruction  In  mathematics  (on  this  general  point,  see 
Adi,  1978;  Crady,  1976;  Lovell,  1972;  Mehlhorn,  1981). 

We  do  not  wish  to  take  sides  on  this  debate,  rather  we  simply  wish  to 
point  out  that  among  the  teachers  we  have  seen,  the  practice  of  teaching 
algebra  as  a  set  of  rules  Is  widespread,  and  Is  often  justified  on 
grounds  of  favouring  "weak*  students  and  on  the  grounds  of  "efficiency"; 
for  dissent  on  these  final  points,  see  (Brown,  1982). 

4.  SUMMARY  OF  EXPERIMENTS  AND  IMPLICATIONS 

The  three  studies  reported  above  were  undertaken  to  help  determine  how 
PIXIE  -  an  ITS  that  has  the  capability  of  diagnosing  errors  and  an 
embryonic  capability  of  remediation  -  might  be  aodifled  to  perform  these 
activities  wore  effectively.  The  following  section  summarises  the  con¬ 
clusions  froa  the  experlaents  and  discusses  their  implications  for 
PIXIE. 

In  summary,  we  say  draw  three  tentative  conclusions  from  these  experl¬ 
aents:  a)  In  the  absence  of  a  student,  teachers  in  Study  A  generally 
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used  a  means-ends  approach  Co  diagnosing  arrora;  amongst  this  group,  the 
•oat  auccaaaful  diagnoatlclan  aaarched  for  patterns;  b)  The  majority  of 
the  taachara  uaed  a  procedural  rather  than  a  conceptual  approach  to 
remediating  algebra;  c)  The  teacher  in  Study  C  did  not  take  a  mal-rule 
at  ita  face  value,  but  probed  the  atudent  to  verify  the  mal-rule  and  to 
determine  lta  cauae ,  and  baaed  remediation  on  thla  combined  Information. 

Implication  for  Pixie:  We  ahall  briefly  addreaa  the  implications  for 
PIXIE  arlaing  out  of  each  of  the  above  pointa:  a)  PIXIE  la  able  to  do  a 
much  more  thorough  analyale  of  the  search  apace  than  (theae)  teachera. 
PIXlE'a  model-generation  createa  model*  with  all  known  mal-rule  varia¬ 
tions  and  all  order-aenaitlve  paira  of  rulea.  Thua,  PIXIE  may  well  be 
expected  to  produce  a  better  dlagnoala  than  a  teacher,  who  might  well  be 
Halted  In  the  number  of  potential  solution  paths  that  could  be  held  In 
memory  at  any  one  time. 

b)  It  la  basically  good  news  for  PIXIE  that  teachera  take  a  predom¬ 
inantly  procedural  approach  to  Instruction  and  remediation  aa  It  Is 
relatively  straightforward  to  produce  a  procedural  explanation  for  a 
atudent.  (It  la  analogous  to  giving  a  trace  of  the  steps  undertaken  by 
PIXIE). 

c)  Probing  the  nature  of  a  student's  misunderstanding  la  likely  to 
require  the  atudent  to  give  an  explanation  for  an  action  In  natural 
language.  Natural  language  Interfaces  seem  only  to  be  effective  If  one 
has  a  well-constrained  domain  in  which  the  vocabulary  la  severely  lim¬ 
ited.  However,  a  natural  language  Interface  to  allow  a  atudent  to 
describe  the  several  ateps  In  a  non-determlnlstic  algorithm  has  been 
Implemented  (Slecman  4  Hendley,  1982). 

The  above  conclusions  and  Implications  are  drawn  tentatively,  given  the 
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nunber  of  teachers  Involved  In  the  etudlee.  In  addition,  work  recently 
completed  by  this  research  group  questions  whether  a  detailed  diagnosis, 
with  or  without  known  causes  for  the  sal-rules,  Is  a  necessary  prere- 
qulslte  for  effective  remediation. 
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The  first  2  pages  of  the  questionnaire  are  Identical  to  those  given  to 
the  teachers.  To  cut  down  on  apace,  we  have  then  merely  Hated  the 
remaining  task-student-answer  pairs.  In  the  actual  questionnaire  each 
task-student-answer  pair  appeared  on  a  separate  page;  each  page  having 
the  same  format  as  the  one  containing  task-student-answer  pair  1.1. 


School  of  Education 
Stanford  University 

Selected  Algebra  Probleas  with  Student  Answers 

The  following  are  examples  of  students'  difficulties  in  working  algebra 
probleas.  Please  work  through  each  problen  to  see  if  you  can  Identify 
the  error  the  student  la  asking  In  each  case.  (Note:  This  Is  a 
genuine  exercise  as  ve  currently  have  no  explanation  for  aoae/aany  of 
the  errors  recorded). 

Thank  you  for  ycur  help. 


D  Sleeaan/Eamonn  Kelly 
6  December  1984 
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Algebra  Problems 


1.1  3+4x-18 

x*6 

A.  What  do  you  think  the  student  is  doing  wrong  to  get  thla  answer? 


B.  How  would  you  remediate  this  problem? 
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1.2 

6+4x-32 

x-8 

1.3 

5+3x-27 

x-12 

1.4 

9+3x-4Q 

x-13 

1.5 

4+7x-39 

*-11 

2.1 

3x-2*+7 

*-6 

2.2 

6x-3x+5 

*-7 

2.3 

5x+4x-ll 

*-10 

2.4 

3x-8x+6 

*-17/2 

2.5 

4x-2x+8 

*-7 

3.1 

2+5x-15 

*-1/25 

3.2 

3+4x-18 

*-18/7 

3.3 

4+6x“20 

x-1/22 

3.4 

5+3x-20 

*-12 

3.5 

3+2x-8 

x-48 

4.1 

2x+3x-10 

*•1,  *-3 

4.2 

3x+2x-ll 

x-2,  x-3 

4.3 

3x+2x-13 

x-2,  x-5 

4.4 

2x+4x-14 

x-4,  x-2 

4.5 

2x+3x-10 

x-3,  x— l 

5.1 

7*+5x-16 

x-2 

5.2 

3*+4x-20 

x-/T 

5.3 

10x+4x-21 

-A 
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Table  _1 

Overall  analysis  of  Che  search  techniques  msed  by  the  4  Teachers 


X  of  Items  searched 

#  of  "complete”* 

#  of  “matched”* 

for  patterns  of  errors 

searches 

searches 

(X  in  parentheses) 

(X  in  parentheses) 

Teacher  1 

26 

15(65) 

6(26) 

Teacher  2 

0 

15(65) 

*13) 

Teacher  3 

47 

17(74) 

5£22> 

Teacher  4 

81 

19(83) 

13(57) 

•Percentages 

(In  parentheses)  based 

on  23  Items. 

Table  2 

Items  for  which  the 

number  of  errors  diagnosed 

did  not  match 

the  number  remediated 

Items  for  which 

Items  in  which 

Items  in  which 

a  'complete"  search  all  errors  were 

partial  remediatii 

path  was  found 

remediated 

was  given 

(out  of  23) 

Teacher 

1 

15 

9  (602) 

6  (402) 

Teacher 

2 

15 

13  (872) 

2  (132) 

Teacher 

3 

17 

13  (762) 

4  (242) 

Teacher 

4 

19 

11  (582) 

8  (422) 

Note:  Table  2  shows  tne  number  of  items  which  were  completely  and  par- 

tially 

remediated.  Deciding  upon  the  actual  numbers  of  errors  within 

Items  Is  a  subjective  judgment,  and  hence  the  numbers  of 
meaningfully  be  quoted. 


errors  cannot 


i 
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Table  3 


A  categorisation  of  the  teachers'  suggestions*  for  remediation 


(As  percentage  of 

total  suggestions 

per  teachi 

Concept-based 

Rule-based 

Other 

Teacher 

1 

20 

53 

27 

Teacher 

2 

50 

20 

30 

Teacher 

3 

0 

98 

2 

Teacher 

4 

28 

56 

16 

•Note:  Concept-based:  auggestlons  of  the  sort  'Discuss  the  idea  of  a 
variable".  Rule-based:  auggestlons  of  the  sort  "Take  the  Xa  to  the 
left-hand  side".  "Other:  classification'  vague  statements  eueh  as 
"Start  over”. 


Table  4 

Referencing  the  student's  work  during  remediation 


Percentage  of 

remedial  auggestlons  that 

referenced  the 

Equation 

Substeps /solution 

No  reference 

Teacher 

1 

73 

0 

27 

Teacher 

2 

10 

0 

90 

Teacher 

3 

18 

60 

22 

Teacher 

4 

18 

0 

82 

Figure  1 


Example  of  a  GPS-type  Search 
of  a  Solution  Tree 


Equation 


Student's 
Incorrect  answer 


*  Tlie  teacher  said,  ‘The  student  ifnored  the  39  and 
vrate  X  *  1 1 
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